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MEXICO: Can mobile tutors improve 
learning in remote schools? 


Achieving inclusive and quality education for all is a global pri- 
ority, and policymakers are still grappling with the best ways 
to ensure the poorest and most marginalized children are in 
school and learning. About 120 million children — many from 


rural areas — are still out of school, despite a recent global 


push for universal access to education. Even when they stay in 
school, children from poor, rural areas often show the lowest 


gains in learning. Recruiting and retaining teachers to work in 


remote areas is a major challenge, and many low and middle- 
income countries rely on members of the community to teach 
local schools. 

In Mexico, the government runs a mobile tutor program 
to send recent university graduates to work in schools in the 
country’s most rural and marginalized communities, where 
teachers are typically members of the community with limited 
education and training. The World Bank’s Strategic Impact 
Evaluation Fund (SIEF) supported an evaluation to measure 
the impacts of the mobile tutoring program in the Chiapas re- 
gion, where a high fraction of the population is poor and indig- 
enous. The evaluation found that the program improved the 
rate at which students transitioned from primary to secondary 
school and parents’ reports of their children’s social-emotional 
skills. Test scores increased as well. A variant of the program 
that extended the amount of training that tutors received both 
before deployment and during their tenure demonstrated 
slightly higher improvements. These results convinced the gov- 
ernment, which had considered shutting down the program, to 
instead continue sending mobile tutors to remote communities 
and to adopt some of the changes that made the program more 


effective. 


Mexico's Chiapas region, where the study took place, is the 
poorest region in the country, according to 2016 state statistics. 
Schools in marginalized communities are run by Consejo Na- 
cional para el Fomento de la Educacion, known as CONAFE, 
a semi-autonomous government agency responsible for provid- 
ing education services in remote communities with fewer than 
2,500 people. CONAFE schools typically have a single multi- 
grade classroom, with an average of 10- 15 students taught by 


one instructor. The instructors are usually secondary school 


graduates who do not have formal teaching qualifications and 
receive only very basic training. They receive a monthly allow- 
ance and can eventually benefit from a scholarship for univer- 
sity studies. 

Students in CONAFE schools generally perform worse 
than those in the regular system. In 2013, 13 percent of 
CONAFE students scored Good or Excellent in the national 
standardized examination, compared with 42 percent in the 


ministry-run schools. 


This brief summarizes the results of a 2018 Endline Report by Francesco Agostinelli, Ciro Avitabile, Matteo Bobba and Alonso Sanchez, “The 
short-term effects of the Mobile Pedagogical Tutors: Evidence from a Randomized Control Trial in rural Mexico.” 
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In 2009, CONAFE started a mobile tutoring program aimed 
at improving the quality of education in these remote and dis- 
advantaged schools. The tutors or Asesores Pedagogicos Itiner- 
antes (Mobile Pedagogical Advisors), known as APIs, are recent 
university graduates assigned to two schools. They are expected 
to spend about two weeks per month in each school providing 
one-on-one tutoring to the students who require the most help, 
making home visits to encourage parental involvement, and 
providing pedagogical support to the local non-professional 
teachers. But the API program, as structured, didn’t lead to im- 
provements in student learning, as assessed in an earlier evalua- 
tion, and the progression rate from primary to lower secondary 
school remained low. 

In 2014, as part of the World Bank project to support 
CONAFE schools, researchers introduced some changes to the 
program as it was implemented nationally to improve the ef- 
fectiveness of the tutors and set up a randomized control trial 
to test the impact of the changed model on students’ learning 
and grade progression. First, they used an API’s ability to speak 
the main indigenous language in the community as the most 
important criterion for assigning APIs to communities. In the 
previous model, 2/3 of tutors assigned to these communities 


did not speak any indigenous language. Second, because par- 


ents had complained about infrequent visits from the APIs, the 
researchers also provided the supervisors of the APIs an increase 
in salary in exchange for a mandatory increase in the number of 
times they visited the communities. 

In another variant of the program, researchers increased the 
training for APIs to two weeks from one week and organized 
bi-monthly peer meetings so tutors could discuss challenges 
and learn what their peers were doing to solve problems. The 
contents (and the costs) of the augmented training were de- 
signed jointly with CONAFE in order to guarantee scalability, 
in case the program proved to be effective. 

The interventions started in September 2014 and contin- 


ued through a second school year ending in the spring of 2016. 


Population at a glance... 


e Communities in the Chiapas pilot were small and 
remote, with 100 inhabitants on average. 

e Sixteen percent had no road and 64 percent lacked a 
paved road. 

e Eleven percent of communities had political conflicts 
(primarily due to the presence of Zapatista rebels). 

e Test scores were substantially lower in community 
schools than the national average. 


Evaluation 


This study used a randomized controlled trial to test the im- 
pacts of the strengthened mobile tutoring program and the 
variant with augmented training. The evaluation took place in 
a sample of 230 schools that had never received the mobile 
tutoring program before. Of these 230 schools, 70 schools were 
assigned to receive the APIs, 60 were assigned to receive the 
APIs with augmented training, and 100 schools were assigned 
to the control group with no mobile tutoring at the time of 
the study. 

Researchers collected no baseline data and instead used 
administrative data to assess balance among the experimental 
groups. Specifically, they used the 2013 results of a national 
exam (the ENLACE), as well as information about school in- 
puts from a census of schools carried out by the Secretariat of 
Public Education and information about communities, includ- 


ing poverty rates, from the national census and the National 


Commission for the Evaluation of Social Policy. 

Researchers conducted follow-up data collection in in the 
spring of 2016. To measure students’ reading and basic math 
ability, two tests were administered: the Early Grade Reading 
Assessment (EGRA) and the Early Grade Math Assessment 
(EGMA), which had been adapted to the Mexican context. Al- 
though these tests are meant for students in the early primary 
grades, children in the community schools often lag behind 
their urban counterparts by a couple of standard deviations, 
and thus require an easier test. Surveyors interviewed 1,930 
children in 3rd to 6th grade and caregivers in 1,050 households 
about their expectations for their children’s education and their 
involvement in their children’s schooling. Caregivers were also 
asked about their children’s behavior. Surveyors also did ob- 
servations of classrooms to document teachers’ pedagogical 


practices. 
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Mobile tutors helped improve students’ literacy 
skills... 


Compared to the control group, students in the schools re- 
ceiving the strengthened mobile tutoring program scored an 
average of 0.135 standard deviations higher on the EGRA 
test, while students in schools receiving mobile tutors with 
augmented training scored an average of 0.227 standard de- 
viations higher. Despite this large difference in the average 
treatment _ effects, 
these impacts can- 
not be statistically 
distinguished from 
each other. When 
the 


investigated 


researchers 
test 
scores further, they 


found 


gains were primar- 


that these 


ily in the areas that 
students were al- 
ready doing well 
in, like reading fa- 


miliar words, dic- 


tation, and reading 
comprehension, as opposed to the areas where students were 
initially weakest, like the recognition of initial sounds and 


letter sounds. 


..-but math skills didn’t improve as much. 


The math performance of students with mobile tutors could 
not be statistically distinguished from scores of their peers 
in control schools. Students who had the mobile tutors 
with augmented training had higher scores than students 
in the control group, though this gain was only marginally 
statistically significant. A breakdown of scores by topic area 
suggests that statistically significant gains only came in the 
most basic task — number recognition — with limited to 
no gains in more difficult tasks like addition and subtrac- 


tion. Children who did not speak an indigenous language 


at home, however, did appear to benefit from the program 
with augmented training, making gains of 0.192 standard 


deviations on the EGMA exam. 


In communities where the tutors received the ex- 
tra training and the bimonthly meetings, children’s 
social-emotional skills also improved relative to 
the control group, based on what their parents re- 
ported. 


The tutors were supposed to help students and parents 
identify the emotional issues that could affect their learning 
outcomes and deal with them. Although the initial training 
didn’t have any specific content related to social-emotional 
issues, supervisors and coordinators encouraged tutors to 
use the peer meetings to also discuss how to deal with chil- 
dren’s emotions. Parents of children in schools served by tu- 
tors in the enhanced training program reported significant 
improvements in social-emotional wellbeing of their chil- 
dren, while parents’ reports in the group that received the 
tutors without the augmented training could not be statis- 


tically distinguished from reports from the control group. 


In schools served by mobile tutors who had re- 
ceived the augmented training, primary school stu- 
dents were more likely to continue to secondary 
school. 


The progression rate between primary and secondary school 
is low in CONAFE schools, with about 60 percent of pri- 
mary school students continuing to secondary school. But 
in communities where the mobile tutors went through the 
more intensive training and support program, the rate rose 
by 14 percentage points over the control group. Children 
who received the program without the augmented training 
saw a smaller increase that was not statistically significant. 
As with the findings for math, these gains in the transition 
to primary school were concentrated among students who 
did not speak an indigenous language at home, suggesting 
that the program may have benefited children who were not 


the most at risk for dropping out before secondary school. 


NOVA: 


The positive changes may have stemmed from im- 
provements in teachers’ pedagogy and greater pa- 
rental involvement. 


Surveyors observed teachers in their classrooms and docu- 
mented their ability to adjust to students’ learning speed, 
how much time they spent on teaching activities, and how 
engaged their students were. The teachers in schools that 
received mobile tutors with augmented training received 
significantly higher classroom practice scores than teach- 


ers in the control group, while teachers who received mo- 


bile tutors with the standard training could not be distin- 
guished from their peers in the control group. Similarly, 
an index capturing the extent to which parents invested in 
their children’s education (as proxied by their aspirations to 
see their children graduate from high school, their helping 
with homework, and meeting with teachers, among oth- 
er activities) significantly improved only for schools that 
received tutors with the augmented training. The study 
didn’t find any changes in how much kids studied or how 
often they attended school. 


@oyaralvcsieye 


Ensuring that children in remote regions have access to 
high quality services is a global challenge. This evaluation 
provides evidence on the impact of a mobile tutor program 
implemented in highly marginalized communities in rural 
Mexico, and the results suggest that mobile tutors can be ef- 
fective in improving learning and children’s transitions from 
primary to secondary school, particularly when the tutors 
receive adequate training and opportunities to troubleshoot 
issues with their peers. The tutors helped teachers improve 


their classroom practices and encouraged more parental 


investment in education. 

Because both interventions tested during the pilot were 
designed within the restrictions of resources available to 
the government agency responsible for remote community 
schools, the tested models were also scalable. As a result, 
the government of Mexico scaled up the program variant 
with the augmented training for the mobile tutors in the 
fall of 2017, a promising step toward creating tangible im- 
provements in learning for disadvantaged children in rural 


Mexico. 


The Strategic Impact Evaluation Fund, part of the World Bank Group, supports and disseminates research evaluating the impact of development projects to help 
alleviate poverty. The goal is to collect and build empirical evidence that can help governments and development organizations design and 
implement the most appropriate and effective policies for better educational, health, and job opportunities for people in low and middle 
income countries. For more information about who we are and what we do, go to: http://www.worldbank.org/sief. 


The Evidence to Policy note series is produced by SIEF with generous support from the British government's Department for International 
Development and the London-based Children’s Investment Fund Foundation (CIFF). 
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